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Abstract 

In this paper, the problem of state and input constrained control is addressed, 
with multidimensional constraints. We obtain a local description of the boundary of 
the admissible subset of the state space where the state and input constraints can be 
satisfied for all times. This boundary is made of two disjoint parts: the subset of the 
state constraint boundary on which there are trajectories pointing towards the interior 
of the admissible set or tangentially to it; and a barrier, namely a semipermeable 
surface which is constructed via a minimum-like principle. 
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1 Introduction 

In this paper, the problem of state and input constrained control is addressed. The ob- 
jective is to describe the admissible trajectories of a constrained control system with as 
much "freedom" as possible, without introducing additional exogenous elements. State 
constraints have mostly been addressed theoretically in the framework of optimal control 
(see e.g. Chapter VI of jT^ : see also 13 [23] for extensions and more modern presen- 
tations). The approach proposed here is complementary, since we focus attention on the 
characterisation and computation of the boundary of the admissible region, namely the 
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subset of the state space where the state and input constraints can be satisfied for all 
times. A distinctive feature of the problem considered in this paper, compared to optimal 
control, is that we are in a qualitative situation, since there is no a priori notion of a 
quantitative objective function to be optimised, nor of a target to reach. 

Related mathematical ideas may be found in the characterisation of other aspects of 
control systems, namely attainable regions j9], reachable sets [22] , invariant sets [3l I25j 
and problems related to optimal control [IH HI] and differential games pS") 17] . Some of 
the results presented in this paper may be also interpreted in terms of viability kernels, a 
major concept of viability theory [2]. In particular, see the work of Quincampoix |20j on 
differential inclusions and target problems. Numerical studies of constrained trajectories, 
extending the work of Quincampoix may be found, e.g., in jS]. 

In contrast with the latter works, by dealing with ordinary differential equations we 
develop a direct mixed topological and geometric approach leading to computable con- 
ditions. We prove a minimum-like principle satisfied by the barrier, the semipermeable 
subset of the boundary of the admissible set. 

The remainder of the paper is organised as follows. In Section [2] we introduce an 
example that provides physical intuition and motivates the problem and the main ideas. 
In Section [3] we more precisely define the problem we want to address, introduce the 
notation and state some assumptions on the dynamical system and the constraints. In 
Section |4] we investigate the topological properties of the admissible set. In Sections [5] 
and [6] we investigate the properties of the boundary of the admissible region and derive 
the conditions under which that boundary intersects the state constraint boundary, which 
we call ultimate tangentiality conditions. In Section[7]we derive the minimum-like principle 
condition satisfied by the barrier and discuss its relationship with notions from optimal 
control theory. Section[8]presents a number of examples of linear and nonlinear constrained 
systems with a complete characterisation of the admissible region. Section [9] presents the 
conclusions of the paper, and some technical auxiliary proofs concerning compactness of 
solutions of differential equations, followed by some of their variational properties and a 
recall of the maximum principle, are presented in Appendices \K\ and [B] at the end of the 
paper. 

2 A motivating example: Single-particle constrained mo- 
tion 

We start our development by considering a simple example, consisting of a particle of mass 
m moving in space with velocity V{t). We also consider that the motion of the particle is 
constrained to remain in some region of the space and that we can apply a force F(t) to the 
particle in any direction, but with the constraint < 1- This scenario is represented 

in Figure [T| An important feature of this problem, that we want to emphasise here, is 
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Figure 1: Constrained motion of a particle. 



that we consider that the constraints (both, in force and position) have to be satisfied for 
all times, and not just over a finite time window. 

The first question is: Given our knowledge of the position X{t) of the particle and its 
velocity V{t) at a given time instant t, what action should we take/start taking, given the 
additional knowledge of the constraint? Are we allowed to move into any region of the 
space, provided we remain (in the figure above) to the left of the constraint surface? 

Consider the limit case represented in Figure [2] It is evident that such a situation 




Figure 2: A "limit situation". 



can not be allowed, since from Newton's second law we know it would not be possible to 
prevent the particle from entering the forbidden region. (Only a force infinite in magnitude 
would be able to instantaneously change the direction of the particle, a situation that our 
constrained input does not allow.) 
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An intuitively simple solution would be to think of a "buffer region" of thickness e that 
would allow our particle enough time to change direction steered by the constrained force. 
As represented in Figure [3j it is to be expected that such a buffer region will depend on 
both, the position X of the particle and its velocity V at that specific position. Many ways 




Figure 3: A "buffer" region. 



of constructing such a "buffer region" could be devised, for example one could construct an 
"artificial potential" such that the particle, once entered into the region, is repelled back 
or, at least, is not allowed to overpass the constraint (see e.g. [5]). By means of influencing 
F we could, for instance, introduce "artificial damping" so as to dissipate the energy of 
the particle and prevent it from trespassing the constraint. However, these — potentially 
useful in practice — ingenious ways of tackling the problem would seem to defeat our initial 
purpose of "parameterising with as much freedom as possible" all the possible constrained 
"natural" trajectories of the system (without adding artificial exogenous elements). 

Still, motivated by the previous considerations, we will analyse one last fact related 
to the simple single-particle example discussed above. It is obvious that, whatever the 
particle's trajectories do in space, the moment they enter in contact with the constraint 
surface they must do so with a velocity that is tangential to the constraints (note that 
these limit trajectories cannot be perturbed without the risk of violating the constraints). 
This situation is represented in Figure [4| 

In general terms, in this paper we prove that such a tangency property is necessary for 
a trajectory to remain on the boundary of the admissible set, namely the set of all initial 
states from which there exists a trajectory satisfying the constraints for all times. We 
further prove a general minimum-like principle to construct this boundary, which arrives 
tangentially to the constraint. 
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3 Constrained dynamical control systems 

We consider the following constrained nonlinear system: 

x = f{x,u), (1) 

x{to) = xo, (2) 

u€U, (3) 

9i(x{t))<0 VtG[to,oo), ViG{l,...,p} (4) 

where x{t) G M"', M" being endowed with the usual topology of the Euclidean norm. We 
denote by Ui = {v € : ||z;|| < 1} the closed unit balj^of M™'. The input function u is 
assumed to belong to the set U of Lebesgue measurable functions from [fojOo) to Ui, i.e. 
u is a measurable function such that < 1 for all t € [to,oo). 

We need to recall from [19j the definition of Lebesgue or regular point for a given control 
u gU: the time t G [tQ,co) is called a Lebesgue point for n if u is continuous at t in the 
sense that there exists a bounded (possibly empty) subset Iq C [to,oo), of zero Lebesgue 
measure, with t Iq, such that u{t) = lims^t,s<^io u{s). Since u is Lebesgue-measurable, 
by Lusin's theorem, the Lebesgue measure of the complement, in [tojT^li for all finite T, 
of the set of Lebesgue points is equal to 0. 

^We keep the same notation || ■ || for the Euchdean norm of W for every p > 1. 
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Note that if ui £U and U2 € U, and if r > is given, the concatenated input v, defined 

by v(t) = I ^^/^^ ! ^ i^o^'^i satisfies v £U. The concatenation operator relative to r 
^ ^ [ ii2(t) if t > r ^ 

is denoted by n,-, i.e. v = ui vir U2- 

To easily handle the multidimensional constraints Q, let us introduce the following 

notations. The constraint set is defined as: 

G^{xGM":5i(x)<0,i=l,...,ri. (5) 

• Assuming that G, endowed with its relative topology in W^, has nonempty interior, 
denoting g{x) = {gi{x) , . . . , gp{x)) , we denote by g{x) = the fact that x G M" 
satisfies gi{x) = for at least one i £ {1, . . . ,p} and gi(x) < for all i £ {1, . . . ,p}. 
The set of all indices i G {1, . . . ,p} such that gi{x) = is denoted by I{x). 

• By g(x) -< (resp. g{x) ^ 0) we mean that gi{x) < (resp. gi{x) < 0) for all 
i G {l,...,p}. 

We also define the sets 

Go = {x G : g{x) = 0}, G_ = {x G M" : ^(x) -< 0}. (6) 

Clearly, the constraint set is given by G = Go U G_. We call the points of Gq boundary 
points. 

We further assume: 

(Al) / is an at least G^ vector field of M" for every u in an open subset of M™ containing 
Ui, whose dependence with respect to u is also at least G^. 

(A2) There exists a constant < G < +oo such that one of the following inequalities 
holds true: 

(i) s^PueUi < C{1 + ||x||) for all x; 

(ii) sup„g^^ |x"^/(x,u)| < G(l + ||x|p) for all x. 

(A3) The set /(x, Ui), called the vectogram in [13j, is convex for all x G M". 

(A4) g is an at least C"^ function from M" to W; and for each i = 1, . . . ,p, the set of 
points given by gi{x) = defines an n — 1 dimensional manifold (also called, loosely 
speaking, a face of the set G). 

In Q, x{t) — sometimes denoted x"(t), or x^'^'^°\t), or even x^"'^'''*''^(t), when the 
distinction is required — denotes the solution of the differential equation ([T]) with input 
u GlA and initial condition Q . 
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Going back to the concatenation operator, it is readily verified that the integral curve 
x^'"'^o,to) generated hy v = ui U2 from xq at initial time to satisfies 

^{v,xo,to) ^ 2,(«i,xo,to) ^(«2,a;("i'"0.*o)(r),T) 

i.e. coincides, on [toiT"]) with the arc of integral curve generated by ui on this interval, 
and, on [r, oo), with the integral curve generated by U2 starting at time r from the end 
point x(''1'^0'*'')(t) of the previous one. 

Several results, concerning compactness of the solutions of system ([T]) and some of their 
variational properties, that are key in various proofs of the remaining sections, and which 
are available in the literature in a rather scattered way and embedded in slightly different 
contexts (see e.g. O [161 [23] ) , a-re presented for completeness in the Appendices [A| and [B] 
at the end of the paper. 

Without loss of generality, since system ([T]) is time-invariant, to may be replaced by 0. 
For simplicity's sake, we adopt this choice in the sequel and, when clear from the context, 
"Vt" or "for almost all t" will mean "Vt G [0,oo)" or "for almost all t G [0,cx))". 



4 The admissible set. Topological properties 

In this section we define the admissible set for the constrained nonlinear system Q-Q 
and study its topological properties. 

Definition 4.1 (Admissible States) We will say that a state-space point x is admis- 
sible if there exists, at least, one input function v & U, such that Q-Q are satisfied 
for Xq = X and u = v. Note that the Markovian property of the system implies that any 
point of the integral curve, x^'"'^\ti), ti € [0, oo), is also an admissible point. The set of 
admissible states is expressed mathematically as: 

A = {x£G:3u£U, g(a;("'^)(t)) ^ 0,Vt}. (7) 

Its complement in G, namely A'' = G \ is thus given by: 

= {xeG:yuel(, 3t< +oo,3i G {l,...,p} s.t. £/i(x("'^)(t)) > 0}. (8) 

From now on, all set topologies will be defined relative to G. 

We discard the trivial cases A = ^ and A'^ = 0. Therefore, in the sequel, we assume 
that both A and A^ contain at least one element. 

In addition to the infinite horizon admissible set A, we also consider the family of sets 
At, called finite horizon admissible sets, defined for all finite < T < +oo by 

At = {x£G:3u£U, g{x^'''^\t)) ^ 0, Vt < T}. 
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Accordingly, its complement in G is given by: 

A^ = {xeG:yuGU, 3t<T,3i G s.t. > 0}. 



Remark 4.1 We stress that the study of the admissible set A in Definition \4.1\ cannot be 
reduced to the one of attainable sets (also called reachable sets). Recall (see e.g. 114\ 
[T8\. \2^) that the attainable, or reachable, set at time t from x, denoted by Xt{x), is the 
subset ofW^ defined by 

Xt(x) = {x("'^)(t):ueZ^}. (9) 

Indeed, one can prove tha^ 

At = {x^W : Xt{x)r\G y^il},yte [0, T]}, ^ = {x G : Xt{x)nG ^ 0, Vt > 0}. (10) 

The set A can also be related to the notion of maximal controlled positively invariant 
set, see e.g. fSj^ and the references therein. We point out that, as opposed to the latter refer- 
ences, our definition is concerned with general measurable controls in the classical context 
of ordinary differential equations, and does not depend on the existence of a synthesizable 
feedback law. 

Proposition 4.1 Assume that (A1)-(A4) are valid. The set of finite horizon admissible 
states. At, is closed for all finite T. 

Proof. The result is an immediate consequence of the compactness result proven in 
Lemma A. 2 of Appendix |A| We sketch it for the sake of completeness. 

Consider a sequence of initial states {xfcjfcgN in At converging to x as tends to 
infinity. By definition of At, for every k £ N, there exists Uk & U such that the corre- 
spondin g int egral curve x*-"*'^* ) satisfies 5(x("'='^'=)(i)) < for all t G [0,T]. According to 
Lemma A. 2 there exists a uniformly converging subsequence, still denoted by x("'='^'=), to 



the absolutely continuous integral curve for some u £U. By the continuity of g, we 

immediately get that g{x^'^'^^\t)) ^ for all t G [0, T], hence x G At, and the proposition 
is proven. ■ 



Corollary 4.1 Under the assumptions of Proposition 4-T the set A is closed. 

Proof. For all < Ti <T2 < oo, we have: 

A = Aoo c At2 c Ati c Ao = G. (11) 

Therefore, A = f]T>o -^T and the result is an immediate consequence of the fact that the 
intersection of a family of closed sets is closed. ■ 



Though nowhere used in this paper, identities ( 10 \ are provided for the sake of completeness. Their 
proof, based on the existence of an absolutely continuous section {{t,x{t)) : x{t) £ Xt{x) n G Vt}, is left 
to the reader. 
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Remark 4.2 Condition (A2) on f is introduced here in order to guarantee boundedness 
and uniform convergence of a sequence of integral curves, which are required in the proof of 
the above proposition (see the details in Appendix^^. However, as the examples shown later 
will illustrate, this (sufficient) condition is generally not needed since many systems that 
do not satisfy it will still have bounded trajectories under admissible controls. Any other 
condition on f ensuring uniform boundedness (see e.g. 110]) will give similar compactness 
results. 



5 Boundary of the admissible set 



We conclude from Proposition 4.1 and Corollary |4.1| that the sets At, for all T > 0, and 
A contain their respective boundaries, denoted by OAt and dA respectively. From now 
on, we will focus on the properties and characterisation of these boundaries, which are 
arguably the most important objects to describe the effect of the constraints Q-Q on 
the time evolution of the dynamical system ([T]). 

To start with, let us adapt the definition of semipermeability (a notion introduced 
several decades ago in differential games by R. Isaacs |13| ) to our context. 

Definition 5.1 (Semipermeability) The boundary dA (relative to B) of a closed set 
A d B <Z M" is called semipermeable relative to B and system |Ip, or just semipermeable 
if non- ambiguous, if, and only if, no integral curve of the system starting in A'^ n B can 
penetrate A before leaving B, i.e. if, and only if, for all xq E A'^ n B, for all u ^ U 

and all t such that the connected arc of integral curve xq, a;("'^o)(t) C B we have that 
G A^. 

Proposition 5.1 For all T > 0, the sets dAx H G- and dA Pi G_ are semipermeable 
relative to G. 



Proof. We directly prove this result for T = +oo. The proof for finite T follows exactly 
the same lines. 

Assume that x G A^ H For every n E we denote by i{u) the first exit time from 
G, i.e. 

t{u) =inf{t G [0,oo) : 3i e s.t. 5i(x("'^)(t)) > 0}. 

Such a lower bound exists for every u ^lAhy definition of A'^. If i{u) = for every u gU, 
then, clearly, x G_. Therefore, there exists u €zU such that t{u) > 0. Let us assume by 
contradiction that for such u there exists i G ]0,t{u)[ such that ^ = x^^'^\i) G A and we 
set u = u u where u is such that g{x^'^'^\t)) ^ for all t >i, which exists by definition 
of A. The corresponding integral curve x^**'^^ is therefore equal to x^'^'^^ for t < i and to 
xi^'i) for t > i. Moreover, since ^(^("'^^(t)) ^ for t < t and c/(x("'^")(t)) ^ for t > t, we 
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conclude that g{x^'^'^\t)) ^ for all t G [0,oo). Therefore, we have proven that x G A, 
which contradicts our assumption that x G H G_. It results that no such i can exist, 
and therefore that no trajectory starting in A^ H G- can penetrate A before leaving G, 
thus proving that dA Ci G- is semipermeable relative to G. ■ 

We now prove the following result. 

Proposition 5.2 Assume that (A1)-(A4) hold. We have the following equivalences: 
(i) X ^ A is equivalent to 

min sup max gi{x^''^^\t)) < (12) 
te[o,oo) *=i.-,p 



(ii) X G A^ is equivalent to 



(Hi) X G dA is equivalent to 



min sup max gi{x^'^''^\t)) > (13) 
i6[0,oo) »=i>-,p 



min sup max g(j(x^"'^^(t)) = 0. (14) 
"e*^ te[o,oo)*=i'--,p 

Proof. We first prove (i). If ic G by definition, there exists u & 14 such that 
c/(x("'^)(t)) < for all t > 0, and thus such that sup^gp^oo) maxi=i,..„p c/i(a;("'^)(t)) < 0. We 
immediately get 

inf sup max gi{x^^'^' (t)) < 0. (15) 
''e"te[o,oo)«=i'---,p 

Let us prove next that the infimum with respect to u is achieved by some u G in order 
to get (12). To this aim, let us consider a minimising sequence Uk € 14 , k £ N, i.e. such 
that 

lim sup max gi{x'^^''''''\t)) = inf sup max gi{x'''^'^\t)) . (16) 

According to Lemma |A.2 in Appendix [Aj with Xk = x for every A; G N, one can extract 
a uniformly convergent subsequence on every compact interval [0, T] with T > 0, still 
denoted by x^"'"^^ whose limit is x^"'^^ for some u £U. The continuity of g implies that the 
sequence (^(x^"*^'^)) uniformly converges to the function g^x^"^'^^) on every compact interval 
[0, T] with T > 0. Therefore, for every T > and every e > 0, there exists k(){T,e) G 
N such that, for every k > ko{T,e) and every i G {l,...,p}, we have gi{x^'^'^\t)) < 
gi{x^'^'"^\t)) + e, Vt G [0,T]; which yields, taking the supremum w.r.t. t G [0,oo) and 
i G {1, . . . ,p} on the right hand side 

5i(a;("'^)(0) < sup max gi(x("'='^)(t)) + e VtG[0,r]. 
te[o,oo) *=i.-,p 
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On the other hand, by the definition of the hmit in (16), for every e > there exists 
ki{e) £ N such that for ah k > A;i(e), we have 

sup max < inf sup max 5i(x^"'^^(t)) + e. 

ie[o,oo) *=iv,p mgw te[o,oo) *=i.-.p 

Thus, for ah k > max(A;o(T, e), ki{e)), we get 

gi{x^^'^\t)) < inf sup max + 2e Vt G [0,r], Vi = 1, . . . ,p. 

«e'^te[o,oo)*=i'--.,p 

However, since the latter inequahty is vahd for any T > and it does not depend on 
k anymore, and since its right-hand side is independent of i, t and T, we have that the 
inequahty holds if we take the supremum of the left-hand side with respect to t G [0, oo) 
and i G {l,...,p}. Using, in addition, the definition of the infimum w.r.t. u, we thus 
obtain that for every e > 

sup max gi{x^^'^\t)) < inf sup max gi{x'^^'^\t)) + 2e 

< sup max 5tj(x("'^^(t)) + 2e, 

iG[0,oo)*=l'-'P 



or, using also (15), that 



sup max 5rj(j;("'^)(t)) = inf sup max gi{x^'^'^\t)) < 0, 



which proves (12). 

Conversely, if (12) holds, there exists an input u £U such that 

sup max < 0, 

te[o,oo)*=i'---,p 

which in turn implies that g{x^'^'^'){t)) ^ for all t > 0, or, in other words, x E A, which 
achieves the proof of (i). 

To prove (ii), we now assume that x G and prove (13). By definition of A'^, for all 
u gU, we have supjg^.oo) ^^^i=i,...,p 9i{^^^'^\t)) > and thus 

inf sup max gi{x'^'"''^\t)) > 0. 

The same minimising sequence argument as in the proof of (i) shows that the minimum 
over u gU is achieved by some u £U and that 

min sup max > 0. 

"6" iG[0,oo)«=l'-'P 
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But the inequality has to be strict since, if min sup max gi = 0, it would 

"e*^ te[o,oo)«=i>-'P 

imply, according to (i), that x G A which contradicts the assumption. Therefore, we have 
proven (13). 



Conversely, if (13) holds, it is immediately seen that x is such that 
sup max gi{x^^'^\t)) > for ah u e U. Since the mapping t H> max gi{x^'^'^\t)) 

tG[0,oo)«=lv,P i=l,...,p 

is continuous, the inverse image {t G [0,oo) : max gi{x^^'^\t)) > 0} is a nonempty 

4 = 1,. 

open subset of [0, oo). Thus, for all u £ U, there exists t{u) < +oo such that 
max gi{x^^'^\i{u))) > 0, and hence x G A''', which proves (ii). 

i=l,...,p 

To prove (iii), since A is closed, x G dA is equivalent to x G ^ and x G cl(^^), the 
closure of A^, which, by (i) and (ii), is equivalent to (12) and (13) (the latter with a ">" 
symbol as a consequence of x G cl(^'^)), which in turn is equivalent to (14). ■ 

Remark 5.1 The same formulas hold true for At, A^ and OAt if one replaces the 
infinite time interval [0,oo) by [0,T]. 

We define the sets 

[dA]o = dAnGo, [dA]_ = dAnG-. (17) 

They indeed satisfy dA = [dA]Q U [dA]_. 

We denote by Lfh{x,u) = Dh{x)f{x,u) the Lie derivative of a smooth function h : 

M along the vector field f{-,u) at the point x. 
The boundary subset [dA]Q is easily characterised. 

Proposition 5.3 The boundary subset [OAIq is contained in the set of points z £ Gq such 
that 

min max g,(z, n) < (18) 
with the notations introduced in Section\^ 

Proof. Since [dA]Q C A, for all z G [5^]q, there must exist u £ U and an inte- 
gral curve x^"'^-* which remains in A for all times. Therefore, at t = 0, the vector 
f{z,u{0+)), where tt(0+) is the right limit of u{t) when t \ 0, cannot point outwards of 
G-, i.e. L f gi{z , u{0+)) < for all i G ^{z); otherwise, if there exists i G I{z) such that 
Lfgi(z,u{0+)) > 0, then the integral curve x^"^'^^ instantaneously leaves G through the 
"face" given by equation g^ = 0. Therefore, maxjg]i(j,) Lj(7j(z, u(0+)) < 0, hence (18). ■ 
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Remark 5.2 The subset of the boundary Gq satisfying (18) is called the usable part of 
the constraint boundary in reference to Isaacs' notion of usable part of a targe^llSl. See 
also e.g. H H^. 

We now turn to the study of the complement of [5^]q in the boundary dA. 

Proposition 5.4 Assume that (Al) to (A4) hold. The boundary subset [dA]_ is made 
of points X G G- for which there exists u £ U and an arc of integral curve x*-"'^-* entirely 
contained in [dA]_ until it intersects Gq at a point x^'^'^\t) such that g{x^'^'^\t)) = for 
some (possibly infinite) i £ [0, +oo). 



Proof. Let x € [9.4] therefore satisfying (14). In particular, there exists u £ U such 

that sup max = 0, and i G [0, +oo) such that max = for 

te[o,oo)«=i'-,p i=i,...,p 

the first time (i.e. max gi {x'^^'^\t)) < for all t < t). Then for an arbitrary to S [0,t[, 

i=l,...,p 

the point ^ = a;^"'^)(to) G G_ satisfies, by a standard dynamic programming argument 

(since x("'^)(t) = x^^'^\tQ + t) for all t > 0), min sup max gi{x'^^'^\t)) = 0. It follows 

"SW tG[0,oo) *=i.-,p 

that ^ G and, therefore, the whole arc of integral curve starting from x G G_ is 

entirely contained in until it intersect^ Go at time i. ■ 



Remark 5.3 Note that the arc of integral curve x^"'^^ of Proposition 5.4 
Go after t for some time and then return to G_ . 



may remain in 



Corollary 5.1 From any point on the boundary [dA]_, there cannot exist a trajectory 
penetrating the interior of A, denoted by int(^), before leaving G_. 



Proof. Let x G According to Proposition 5.4, there exists u £ U such that 



(t) G [dA]_ for all t G [0, t[, where t is the first time such that x'''^'^>{t) G Gq. Given 



T G [0,t[, we denote ^ 



„{u,x) 



(r). Assume, by contradiction, that there exists u £U and 



a G [0,e[, with e > small enough so that x*^"'^)(r + t) G G_ for all t G [0,e[ and that 
C — x^"'^^(t + cj) G int(^). As a consequence of (12) and (14), there exists v £ Li such 

that sup max gi{x^'"''^\T + a + t)) < 0. Setting v = u Xr u xIt+o- v, we easily verify 

te[o,oo)*=i'-'P 

that sup max gi{x^'"'^\t)) < 0, which implies, again by (12) and (14), that x £ int(^), 
te[o,oo)«=i'---,p I I I 

hence contradicting the fact that x £ [dA]_. We thus conclude that no integral curve 

starting in can penetrate the interior of A before leaving G_. ■ 

^Note that, in our context, the boundary Go of the constraint Q cannot be interpreted as a target 
since we do not a priori want the trajectories to finish on Go- On the contrary, Go may be considered as 
a target for the non admissible trajectories (starting in A^) and the usable part of this target is given by 
the relation min^gyj max^gij^) Lfgi(z,u) > 0, opposite to (181. 

*If +00, the whole integral curve a;'"'^^ remains in G- and satisfies limt_^+oo x^"'^\t) £ Go- 
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6 Ultimate tangentiality conditions 



As concluded from Proposition |5.4| the boundary subset [dA] _ is made of integral curves 
intersecting Gq. In this section, we provide a precise characterisation of how this inter- 
section occurs. We will first deal with the particular case of a scalar state constraint and 
then we extend the result to the general multi-dimensional constraint case. 

6.1 The scalar state constraint case 

We consider here the case p = 1 in Q , hence the maximum over { 1 , . . . , p} in all previous 
expressions is not necessary. 

Proposition 6.1 There exists a point z = x^^'^\t) € cl([c?.4]_) n Go, i.e. such that 
g{x^^'^\t)) = for some finite time t > and satisfying 

min Lfg{z, u) = 0. (19) 
u&Ui 

In other words, at such an intersection point z, there exists u £ Ui such that the vector 
field f{z,u) is tangent to Gq at z, and Lfg{z,v) > for all v G Ui, i.e. the vector field 
f{z,v) points outwards of G- at z for all v GUi. 

Proof. Consider x G C G- and u GlA such that the integral curve x^'^'^\t) G 



for all t in some time interval until it reaches Gq (as in Proposition 5.4) and, consider 
further that u GlA \s prolonged in such a way that the integral curve remains in G for all 
times (which is possible by the definition and closedness of A). 

Let us introduce a control variation u^.e of u of the form ( |52[ ) , where r is an arbitrary 
Lebesgue point of u before the integral curve x^"'^^ intersects Gq. 

Since x € there exists an open set O C such that x + e/i G A'^ for all 

h £ O and \\h\\ < H, with H arbitrarily small, and all e sufficiently small. Therefore, 
by definition of A^, the integral curve t i— )• ^("'^■^'^"'"^''^(t) must leave G_ and crosses Gq 



at some t{K,£, h) < oo. Moreover, according to Lemma B.l, the family of integral curves 
^(uK,j,a;+£/!.)^ indexed by e, converges to x*-"'^^ as e — t- 0, uniformly with respect to t, k and 



h. Consequently, with the notations of Section B.l, the set 



{t{K,£,h) : K £ Ui X [0,T] X [0,L],£ e [0,eo], WH <H,heO} 

is uniformly bounded and there exists a continuous scalar function p such that lim^-^o /o(e) = 
and 

-p{e)<g{x^'''^Ht{'^,e,h)))<p{e) 

for all £ and \\h\\ sufficiently small and all k. As a consequence of the uniform convergence 
of x^'^'^'^'^'^^^^ to x*^"'^^ and the continuity of the function t ^ g o x^'^'^\t), there exists 
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some finite t satisfying lim£_5.o5(x*^"'='='^"'"^'^^(t(«;, e, /i))) = = g{x^'^'^\t)). We denote 
By tlie definition of t{K,e,h) we have g(a;^""'^'^^^^-'(t(K, e, /i))) = 0, and by the defi- 



A 

z = X 



nition of u we have g{x^^'^\t(K,£,h))) < 0. Thus, according to (54) of Appendix |b| we 
have 

= eDc/(x("'^)(t(K,e,/i)))i(;(f(K,e,/i),K,/i) + 0(e2) > 



Therefore, using (55), after division by e, 

e, /i))) [«>"(t(K, e, /i), 0)/i 
+ l$"(t(K,e,/i),r) (/(x("'*)(r),t;) - /(x("'^)(r), u(t)))] + 0(e) > 0. 

Letting \\h\\ and e tend to 0, dividing by /, and using again the uniform convergence, 
we readily get: 

Dgix^^'^'Hm^ilr) (/(x(^'^)(T),i;) - /(x(^'^)(r), u(r))) > 

for all V ^ Ui and for all Lebesgue points t <i. 

Thus, if t is a Lebesgue point of u, setting r = t , we have, using the Lie derivative 
notation, 

Lfg{x^^'^\t),v) > Ljg{x(^'^\t),u{t)) 

for all w € C/i, i.e. 

Lfg{z, u{t)) = min Lfg{z, v). (20) 
f e(/i 

Otherwise, if t is not a Lebesgue point of u, choosing r arbitrarily close to t, since 
$"(t, r) is arbitrarily close to the identity matrix, the same conclusion holds true by 
replacing u{t) by the left limit U-{t) = lim^_i.f u(r). It suffices then to modify u, 



without loss of generality, on the 0-measure set {t} by u-{t) to obtain (20). 



Finally, since z G [9.4]q, according to Proposition 5.3, we have uiiny Lfg{z,v) < 0. 
But, since the mapping is non decreasing in an interval ]t — 5,t\ with 

5 > small enough, we also have < Lfg{x^'^'^\t),u^{t)) = miny Lfg{z,v) and thus 
min^, Lfg{z, v) = 0, which achieves the proof of the proposition. ■ 

6.2 The multi-dimensional state constraint case 

We now consider the general case of multiple p state constraints in (|4]). 
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Proposition 6.2 There exists a point z = x^^'^\t) G cl([(9^]_) n Go, i.e. such that 
c/(x("'^')(F)) = for some finite time t > and, with the notations introduced in Section^ 

min max Lfafz, u) = 0. (21) 

In other words, there exists a point z at the intersection between Gq and an arc of integral 
curve in the boundary [dA]_, and n G C/i such that the vector field f{z,u) is tangent at 
z to at least one face defined by gi{x) = 0, for i £ I{z), and does not point outwards with 
respect to any face corresponding to I{z), i.e. Lfgj{z,u) < for all j £ ^{z). In addition, 
for all V £ Ui, the vector field f(z,v) points outwards of G- at z, i.e. there exists j £ I{z) 
such that Lfgj{z,v) > 0. 

Proof. Consider x £ [dA]_ C G- and a finite t > 0, obtained as the limit of the crossing 
times associated to a sequence of variations as in the proof of Proposition 6.1, such that 

max = (22) 

i=l,...,p 

and denote z = x^'^'^\t). 

By the same argument as in the proof of Proposition 6.1 , each element of the perturbed 
sequence leaves G_ through one of the faces of G, say i = i{K,e,h) £ I{z), 

of equation gi(K,e,h)i^) = 0- Choosing a subsequence of {k, for e and h sufficiently 
small, such that i{K,e,h) = zq for some fixed £ I(-z), we readily deduce from the proof 
of Proposition |6.1| that 

"oiin Lfgi^,{z,u) =0. (23) 



Taking into account that z £ [dA]Q, the result of Proposition 5.3, says that 



= min Lfgig{z, u) < min max Lfgi{z, u) < 



which immediately gives (21). 



7 The barrier equation 

For topological reasons, we now restrict our attention to the part of the boundary [f?^]_ 
which intersects the closure of the interior of A, denoted cl(int(^)), assuming that int(^) ^ 



Theorem 7.1 Under the assumptions of Proposition 5.4- every integral curve x"^ on 
[dA\_ric\(^\nt{A)) and the corresponding control function u, as in Proposition 5.4. satisfies 
the following necessary condition. 
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There exists a (non zero) absolutely continuous maximal solution A" to the adjoint 
equation 

= - (|^(x^(t),n(t)))^A^(t), \^{t) = {DgAz)f (24) 

such that 

min {{X^{t)ff{x^{t),u)} = (A^(t))'^/(xnt), = (25) 
at every Lebesgue point t of u (i.e. for almost all t < i). 



In (24), t denotes the tim^at which z is reached, i.e. x"(t) = z, with z G Go satisfying 



gi{z)=0, i£l{z), min max Lfgi{z,u) = Lfgi*{z,u*) = 0. 

u&Ui iei{z) 

Moreover, A"(t) is normal to [dA]_ D cl(int(^)) at x'^{t) for almost every t <t. 

Before proving this theorem, we provide several comments and we recah that the 
attainable set Xt{x) from a point x € M" at time t is the subset of defined by ([o]). As 
a direct consequence of Lemma A. 2 of Appendix |A| Xt{x) is compact for all finite t. We 
denote by dXt{x) its boundary. 



Remark 7.1 Theorem 7.1 may be interpreted as follows: Setting 

H{x,\u) = \^f{x,u) (26) 
n cl(int(^)) is made of trajectories 1 1— )• x^{t), which are projections by 

vr : (x, A, n) I— x = 7r(x, A, u) 



of trajectories of the triple {x'^ , , u) , solution to the Hamiltonian system 



= (x"(t),A^(t),u(t)), A^(t) = - — (x^(t),A^(t),n(t)) 



(27) 



/?(x"(t),A"(t),u(t)) = mini/(x"(t),A"(t),n) =0 

computed backwards in time from the final condition 

(x"(t),A"(t),n(t)) = {z,{DgAz)f .u*) G Gq x M*^ x Ui, 

such that the set of equations 

gi{z)=0, i£l{z), min max Lfgi{z,u) = Lfgi*{z,u*) = (28) 

u&Ui fiei(2) 

admits a local solution. 
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Figure 5: Each point of the boundary, ^ S ncl(int(^)), also belongs to the boundary 

of the attainable set, dXt{x), from x G [dA]_ H cl(int(^)) at every sufficiently small time 
t, and the vectogram /(^, Ui) points outwards with respect to A and inwards with respect 
to Xt{x). In other words, A, the normal to Pi cl(int(^)), is such that A"^/(^,u) > 

for all u £ Ui. 



Remark 7.2 The necessary conditions (21)- (28) may he compared to the one obtained by 



Isaacs in the context of harriers in differential games 1 13] and, of course, to the Pontryagin 
maximum principle (PMP) of fl9j in the context of optimal control. Note however that 
in our case, contrarily to those references, there is no a priori cost function to optimise 



(note that, even though the characterisation of the admissible set (12) may be interpreted 
in terms of a minimisation problem, the cost function is not a standard one). Moreover, 
as far as the constraints cannot be directly interpreted as a separate player, there is no 
game involved. 

However, it has been remarked since long (see e.g. /76[ Chapter 4, P- 239], fJi Sec- 
tion 10.2, p. 136]) that the maximisation of the Hamiltonian in the PMP was in fact 
characterising the boundary of the attainable set from prescribed initial conditions. More 
precisely, at every boundary point of the attainable set, no vector of the tangent pertur- 
bation cone (see fiy\. {^), and in particular of the vectogram of admissible directions, 
can point outwards the attainable set, i.e. the scalar product of any such element of the 
perturbation cone with the normal vector to the boundary constituted by the adjoint is non 



positive (see Theorem B.l in Appendix]^. In addition, the adjoint vector is obtained by 



'Note that t can be chosen arbitrarily because of the time-invariance of the problem. 
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parallel transport of the final conditions, the so-called transversality conditions, along an 
optimal integral curve (Equation (58)). Here, we are in a similar situation, as depicted 
by Figure^ the only difference with respect to the PMP being that on [dA\_ ncl(int(^)) 
the adjoint vector A we consider is an outer normal to [dA\_ H cl(int(^)) and therefore 
a normal pointing inwards the attainable set, opposite to the one considered in the PMP, 
hence the minimum replacing the maximum with respect to u (see Equation 



'25)) 



We need the following proposition. 



Proposition 7.1 Let x € n cl(int(^)) and u £ U as in Proposition 5.4. i.e. such 

that x^^'^\t) € n cl(int(^)) for all t € [0, t[, where t is the first time such that 

max =0. 
1=1,. ..,p 

Then x^'^'^\t) £ dXt{x) for all sufficiently small < t < t. 

Proof. We first prove that Xt{x) C cl(^'^) for all sufficiently small t. Since x G G-, 
by the continuity of the attainable set Xt{x) with respect to t (see [H]) there exists a 
small enough time interval in which Xt{x) C G_. It follows from Corollary 5.1 that 
Xt{x) n int(^) = 0, thus, Xt{x) C c\{A'^). 

Thus, by complementarity, int(^) C Xt{x)^, and consequently cl(int(^)) C c\{Xt{x)^). 
Therefore x^^'^\t) G n cl(int(^)) C c\{Xt{x)^). Since, by definition, we also have 

x^'^'^\t) G Xt{x), the conclusion follows from the fact that dXt{x) = Xt{x) D c\{Xt{x)^). 



The proof of Theorem 7.1 , is based on the necessary condition for a point of the state 



space to belong to the boundary of an attainable set given in Theorem B.l of Appendix [B] 
( [161 Theorem 3, Chapter 4, p. 254] or [U Theorem 12.1, p. 164 and Theorem 12.4 p. 178]. 
See also [HIM])- 



Proof of Theorem |7.l[ Let x and n be as in Proposition 7.1 Denote xi 



X2 



E("'^')(ti) and 

(t2) for sufficiently small ti and t2 such that xi G dXtj^{x) and X2 G dXt^ixi) 



as in Proposition |7.1[ By Theorem |B.l , there exists an absolutely continuous maximal 
solution r]i (resp. 772) satisfying Equations (58)-(59) on the interval [0, ti] (resp. [ti,i2])- 



By the homogeneity of Equation (58), the end-point conditions can be chosen so that 
i]i{ti) = r]2{ti), thus achieving the same constant in (59) 
77" defined on [0, t2] such that r/" = r/" on [0, ti] and ry" = 

argument, the solution 77" can be extended to any subinterval of [0, t]. Denoting A" 



It follows that there exists 
772 on [ti,t2]- By the same 



-77" 



it is immediately seen that A" satisfies (58) and that the max operator in (59) is replaced 
by min. 
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Moreover, the final condition of A" at t is given by Proposition 6.2 hence (24), which 



implies that the constant of the right-hand side of (59) is equal to 0. We have thus proven 



(24) and (25), which achieves the proof of Theorem 



7.1 



8 Examples 

We present a number of examples to illustrate the use of the results of the paper. 
8.1 Linear spring 

Consider a system consisting of a mass and a spring, governed by the differential equation 
mij+by+ky = u, where m is the mass, y the displacement, b the (linear) friction coefficient, 
k the spring constant and u the force applied to the mass. We consider a state constraint 
as in ^ of the form y{t) < xi, where xi is a constant position that must not be exceeded, 
and a constraint in the input as in ([s]), i.e., \u{t)\ < 1. By introducing the variables xi = y, 
^2 = the system and constraints can be written as: 



Xl 
X2 







1 



Xl 
X2 



+ 





m 



n. 



\u\ < 1, Xl — Xl < 0. 

From the ultimate tangentiality condition on Go [i.e., on (xi, X2) = (xi, X2), X2 G M], given 



by (19), since Dg{x) = (1,0), we have: 



= min {Dg{x)f{x,u)} = min {X2} = X2- 

\u\<l \u\<l 

Hence, the point (2:1,0) of Go is the endpoint of the trajectory in cl([9^]_) that arrives 



tangentially to Go (see Propositions 5.4 and 6.1) 



The rest of the points belonging to the barrier [dA] _ are given by equation ( 25 ) , 

min f(x,u) \ = 0, 

\u\<l 1. ^ ' 



and satisfy equation (|24|), that is, 
A = 



Ai 
A2 



-{x,u)) A 



^ 

m 

-1 A 
m 



Ai 
A2 



Xi{t) 



(29) 



where t is the time of arrival at the endpoint {xi, 0). The trajectory X(t) of the adjoint sys- 



tem can thus be obtained by backward integration of (29). In addition, from equation (25) 
we have: 

L , , k b 1 .] , , 

min <^ A1X2 + A2( xi X2 -\ n) ^ = 0, (30) 

|n|<i L m m m ) 
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from where we deduce that, on the barrier, u = — sgn(A2) and that the barrier points are 
described by: 



Xl 
X2 







1 



Xl 
X2 



1 ) sgn(A2), ' 



X2it) 



Xl 





(31) 



The trajectory x(t) determining the barrier can thus be obtained by backward integration 



of (31). The resulting trajectory that gives the barrier of the admissible set A is shown 



in Figure |6] for the numerical values m = l, k = 2, b = 2 and xi = 1. 




-40 -35 -30 -25 -20 -15 



Figure 6: Admissible set A and barrier for the linear mass-spring example 



8.2 Nonlinear spring 

We next consider a nonlinear version of the spring, in which the force exerted by the 
spring is given by k{xi + xf) instead of the linear force kxi and where, as before, xi = 
y = displacement (this situation is commonly referred to as hardening spring). From an 
analysis almost identical to the one performed previously for the linear spring, we conclude 
that the barrier points can be obtained from the following system and adjoint equations: 



J ^1 = ^|(l + 3xf)A2 \ ±1 = X2 

IA2 = -Ai + ^A2 '1*2 = -A(xi + x?)-A,,_i.,gn(A2) ^''^ 
with endpoint given by {\i{t),X2{t),xi{t),X2it)) = (l,0,xi,0). The trajectory determin- 



ing the barrier can thus be obtained by backward integration of system (32) starting from 
the endpoint. The resulting trajectory that gives the barrier of the admissible set A is 
shown in Figure [7] for the same numerical values as before. For comparison purposes, 
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spring: 






spring: kx^^ / 












spring: fc (xj^ + 


^) 

200^ 





40 1 1 ' ' 1 1 1 1 i-i 1 ' 

-40 -35 -30 -25 -20 -15 -10 -5 5 10 



Figure 7: Admissible sets A and barriers for the non-linear mass-spring examples 



also shown in the figure are the cases of the linear spring kxi and an intermediate case 
k{xi + 2oo)- Note, from the figure, the considerable reduction in the size of the admissible 
set due to the effect of the hardening spring; namely, a hardening spring is able to store 
more potential energy and, hence, it can surpass the position constraint if started from 
initial conditions that would not cause constraint violations in the case of a linear spring. 

8.3 A nonlinear academic example 

We consider the 2-dimensional single input system: 



xi = 1 — X2 
X2 = u 



(33) 



with input constraint 

\u\ < 1 (34) 

and state constraint 

a<xi<a, a, aGM, a<a. (35) 

Denoting as usual x = {xi,X2), we have G = {x G : a < xi < a}. We set g{x) = a — xi 
and g{x) = xi — a. Clearly, G is given by the set of inequalities 

g{x) < 0, g{x) < 0. 

The boundary Gq of G is thus made of the union of the sets a — xi = and xi — a = 0. 
Since the latter sets define two disjoint 1-dimensional smooth manifolds (parallel straight 



22 



lines), a normal vector to Go is given by Dg{x) = (—1,0) \i xi = a and Dg{x) = (1,0) if 
xi = a. We thus consider the two cases, where the active constraint is xi = a or xi = a, 
separately. 

8.3.1 The case xi = a 

The ultimate tangentiality condition (I2T|) reads 



min{-l(l - xl{t)) + • u} = 
|«|<i 

where t is a time instant where an arc of the system integral curve contained in [f?^]_ 
intersects Gq = {x G : xi = a}. We readily get x^t) = 1. For convenience, we denote 
X2{t) = X2 and we have X2 = ±1. 



The Hamiltonian (26) is H{x, A, u) = Ai(l — X2) + A2it, where A = (Ai, A2) is the adjoint 



state, and the adjoint equation is 

Ai = 0, A2 = 2X2A1 

with final condition at time t 

Ai(t) = -1, A2(t) = 
The adjoint is thus given by Ai(t) = — 1 for all t < t and 

A2(t) = -2x2{t), X2{t) = 0. 



Moreover, we must have, according to (27), 



min H{x{t),\{t),u) = H{x{t), X{t),u{t)) = 0, a.e. t < t 

\u\<l 

or u{t) = — sign(A2(i)) and —1 + X2{t) = \X2{t)\. Since the r.h.s. of the latter expression is 
non negative, we deduce that either X2{t) > 1 or X2(t) < — 1. 

Setting A2 = sign(A2(t)) for t in an interval ]ti,t[ to be determined, the arc of curve of 
[<9.4]_ arriving at (a, X2) is given by 

xi{t) = I - xl{t), X2it) = -X2 

with 

xi{t) = a, X2{t) = X2 

and with 

A2(t) = -2x2{t), X2{t) = 0. 
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We readily get: 

X2{t) =X2 - A2(t-i) 
and, since X2 — =^1 and A2 = ±1, 1 — X2(t) = 2j;2A2(^ ~ t) — {t — t)'^, and we have 

xi{t) = a-^{t-tf + X2X2{t-tf = a-^{t-tf{t -t -3x2X2) 

and, according to A2 = 1, 

A2(t) = -2x2(t -t) + \2{t - tf = \2{t-t){t-t- 2X2X2) . 



Since xi{t), given by (37), has to satisfy the constraint a — xi{t) < 0, we have 

a-xi{t) = ^{t- {t-t- 3x2X2) < 

Vt which, together with the fact that |(t — t)^ > 0, yields t — t — 3X2A2 ^ 0; 

t-t 

= sup — — < X2A2. 

ii<t<t o 

Thus, since X2 = ±1 and A2 = ±1, we get X2A2 = 1 and, 

X2 = A2- 



Thus, (36), (37), (38) read 



X2{t) = -A2(t-t-l) 

2;i(t) = a - -(t - €f {t-t- 3) 

A2(t) = A2(t-t) (t-t -2). 



Finally, eliminating t-t from the first equation of ( 39 ) , we get 

X2 



and 



xi = a + - 1 



t - t = 1 



X2 



A, 



2 + 



X2 



Thus, if A2 = 1 (which implies u = —1): 



1 2 
xi = a + -{I-X2) (2 + X2) 



an expression valid for X2 > 1 (since t < t). 
Now, if A2 = —1 (thus u = +1): 



1 2 
xi=a + -{I + X2) (2 - X2) 



an expression valid for X2 < — 1 (since t < t). 
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8.3.2 The CEise xi = a 

The only difference with the previous case is that Ai(t) = +1 (the first component of the 
differential of g at xi = a) for t <t, where i is a time instant where an arc of the system 
integral curve contained in intersects the line xi = a. The ultimate tangentiality 

condition reads here min|„|<i{l — ^2 + O.u} = 0, or X2 = ±1. 

The Hamiltonian H{x, A, u) = Ai(l — x^) + Mu is unchanged, as well as the argument 
of its minimum with respect to u, denoted by u{t), u{t) = — sign(A2(t)). The adjoint 
equation, with Ai(i) = +1, reads 

Ht) = 2x2{t), X2{i) = 0. 

Setting A2 = sign(A2(i)) for t in an interval ]t2,i[5 the integration of the system gives 

X2{t) =X2 — X2(t — i) 

xi{t) = a-\{t-tf [t- 1-3x2X2) 

X2{t) = -Xiit -t)[t-t- 2x2X2) 

Since we must have xi{t) < a, the expression of xi yields t — t — 3x2X2 > 0, or 
X2X2 < infj^^^^^ ^ < and, thus, X2A2 = —1, i.e. X2 = — A2, and t — 3 < t <t. 
Thus the previous expressions of X2(t), A2(t) read: 

X2{t) = -A2(t-t+l) 

xi{t) = a-\{t-tf{t-t + 3) 

X2{t) = -X2{t-t) {t-t + 2) 

Elimination of i — i in the first equation yields t — t = —1 — and 

Thus, if A2 = +1 (which implies u = —1), we get 

Xi{t) = a-^{1 + X2f{2-X2) (42) 
and if A2 = — 1 (which implies ?Z = +1), 

xi{t) = a-^{l-X2f{2 + X2). (43) 
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Figure 8: Admissible set A and barrier for system (33)-(35) with a = — 1 and a = 3 



Remark that the two arcs (42) and (43) cross at the point (xi,X2) = (a — 1,0). At 



this point, if a < a — |, two controls M = +lorti = — 1 are allowed in order to remain in 
the admissible set. This means that the barrier (made of the pair of corresponding arcs of 
integral curves from (a — |, 0) to (a, ±1)) is not differentiable at the point (a — |, 0) where 
the normals are orthogonal, (Ai,A2) = (1,±1). 

Finally, it is readily seen that the admissible set A is the union of the four subsets: 



A ={{xi,X2) G : a + - (1 + X2) (2 - X2) <xi<a,X2< -1} 

o 



1 



U{{xi,X2) eR^ ■.a<xi<a- -{1 + 2:2) (2 - X2) , -1 < X2 < 0} 
U{{xi,X2) eR^ ■.a<xi<a-\{l- X2f (2 + X2) , < X2 < 1} 
U {(2:1, 2:2) G : a + ^ (1 - X2f (2 + X2) < xi < a, X2 > 1} 



as depicted in Figure [8] for a = — 1 and a = 3. 



Note that the admissible set A is not even a connected set when a > a — |. 
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9 Conclusions 

This paper has addressed the problem of state and input constrained control for nonlinear 
systems with multidimensional constraints. In particular, the admissible region of the state 
space, where the state and input constraints can be satisfied for all times, was studied. A 
local description of the boundary of the admissible region of the state space was obtained. 
This boundary is made of two disjoint parts: the subset of the state constraint boundary 
on which there are trajectories pointing towards the interior of the admissible set or 
tangentially to it; and a barrier, namely a semipermeable surface which is constructed via 
a minimum-like principle. A number of examples was provided to illustrate the results of 
the paper. A complete characterisation of the admissible region was obtained for these two- 
dimensional examples. While the theoretical necessary conditions derived in this paper 
hold for any dimension, higher dimensional systems may exhibit more complex geometric 
features and such examples will be the subject of future study. 

Acknowledgement. The authors wish to express their warm gratitude to Prof. Em- 
manuel Trelat for fruitful discussions. 

A Compactness of solutions 

We prove here the inequalities and compactness results used in Sections |4] and [5] Many 
related results exist in the literature; see e.g. |23l Theorem 5.2.1] for a result based on 
purely functional analytic arguments, and [6l Chap. 9], [El Section 4.2], [l] Section 10.3] 
for related results specifically oriented to the study of attainable sets and the existence of 
optimal controls, based on Filippov's theorem |ilOj . Those results are scattered in several 
sources and embedded in slightly different contexts than the one of concern here, and are 
not always easily identifiable. Thus they are provided here for the sake of completeness 
and unification. 

We recall the following classical lemma: 

Lemma A.l If assumptions (Al) and (A2) of Section^ hold true, equation |Ip admits 
a unique absolutely continuous integral curve over [to,+oo) for every u £ U and every 
bounded initial condition xq, which remains bounded for all finite t > t^, 




(44) 



with a = 1 for condition (A2.i) and a = 2 for condition (A2.ii). 
Moreover, we have 




(45) 
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for all t, s G [to, T] and all T > Iq, where 



C{a) = sup ||/(x, n)|| < +00 

\\x\\<K{a,T),u&Ui 

with a = 1 (resp. a = 2) if condition (A2.i) (resp. (A2.ii)) holds. 



Proof. 



In the case (A2.i), using the integral representation of ([T]), we get 



koll < / ||/(x(T),n(r))||dr 
rt 



< [ C{l + \\x{T)\\)dT according to (A2.i) 



Thus, 



(1 + ||x(i)||) < (1 + llxoll) + C (1 + \\x{T)\\)dT 

Jto 



and, by Gronwall's Lemma [T2]. 

(l + ||x(t)||)<(l + ||xo||)e^(*-*o) 



which readily yields (44) with a = 1. 



In the case (A2.ii), multiplying both sides of ([l]) by we get 

1 d 



2dt 



x{ty f{x{t),u{t)) 



or, in integral representation, and taking absolute values. 



\xit)f-\\xo\ 



x{Tff{x{T),u{T))dT 



to 



<2C f (1 + \\x{T)f)dT according to (A2.ii) 



or 



(1 + \\x{t)f) < (1 + llxof ) +2C [ (1 + ||x(r)f) dr. 
As before, by Gronwall's Lemma, we get 



1 + llx 



< 1+ kor e 



2\ „2C(t-to) 



which readily yields (44) with a = 2. 
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To prove Inequality (45), let us recall that, for every pair t,s £ [to,T] and all T > Iq, 



x{t) — x{s) = f{x{T),u{T))dT. The continuity of / implies that C{a) < +00 with C(a) 



defined by (46). We immediately deduce (45). 



In the following results we will, without loss of generality (due to time-invariance) , 
replace the initial time to by 0. 

Corollary A.l Let us denote by X(xo) the set of integral curves issued from an arbitrary 
xq, \\xo\\ < 00, and satisfying 

If assumptions (Al) and (A 2) of Section^hold true, ^^(^0) is a subset of C^{[0, 00), W^), 
the space of continuous functions from [0, 00) to M", and is relatively compact with respect 
to the topology of uniform convergence on C^{[0,T],W^) for all finite T > 0. In other 
words, from any sequence of integral curves in X{xq), one can extract a subsequence whose 
convergence is uniform on every interval [0,T], with T > and finite, and whose limit 
belongs to C°([0, 00), M"). 

Proof. In Lemma [a. 1 inequality (44) means that the restriction of the integral curves of 



Af(xo) to any finite interval [0, T] is equibounded, and ( |45[ ) shows that the same restriction 
to any finite interval [0,T] of the integral curves of X{xq) is an equicontinuous set with 
respect to the topology of uniform convergence on C'^([0, T], M"), for all T > 0. The 
relative compactness results from Ascoli-Arzela's theorem (see e.g. ^26. Chap. Ill, §3, p. 
85]). ' ■ 

We now prove the following: 

Lemma A. 2 Assume that (Al), (A2) and (A3) of Section^ hold. Given a compact set 
Xq o/M", the set X = IJi^oeA'o '^(^0) is compact with respect to the topology of uniform 
convergence on C^{[0,T],M."-) for allT > 0, namely from every sequence C X 

one can extract a uniformly convergent subsequence on every finite interval [0,r], whose 
limit ^ is an absolutely continuous integral curve on [0,oo), belonging to X. In other 
words, there exists x £ Xq and u £U such that ^(t) = x^'^'^\t) for almost all t >0. 



Proof. Since Xq is compact, it is immediate to extend inequalities ( 44 ) and ( 45 ) to integral 
curves with arbitrary xq £ Xq by taking, in the right-hand side of (44), the supremum over 
all Xq £ Xq. Thus, by the same argument as in the proof of Corollary A.l, using Ascoli- 
Arzela's theorem, we conclude that X is relatively compact with respect to the topology 
of uniform convergence on C'^([0, T], M"), for all T > 0. It remains to prove that from 
every sequence {x^"'"^*^}fcgN C X one can extract a uniformly convergent subsequence on 
every finite interval [0, T] whose limit ^ belongs to X. 

To this aim, we first remark that, from the fact that X is relatively compact we have 
that the limit, ^, is a continuous function on [0,r] for all T > 0, and that, for every finite 
T and every t £ [0,T], 



rt ft 

^(t) = lim Xfc + lim / /(x("'='^'=)(s),nfc(s))ds = x+ lim / Fk{s)ds (47) 

k—^oo Jq k—^00 Jq 
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where the hmit is taken over a subseQiience and. with the notations x — hni/^^oo Xk and 

We denote hy < v,w >= X]F=i''^«^« scalar product of the vectors v and w in W^. 
Since, for every k, the integral curve x^"'"^''^ satisfies x("fc'^fc)(t) = Fk{t) for almost every t, 
taking the scalar product of both sides by a function ip £ C°^([0, oo),M") and integrating 
from to T, yields 

T j-T 

< >dt= < ^{t),Fk{t) > dt 

Jo 

or, after integration by parts: 

- / < > dt+ < (/p(r),x("'='^'=)(r) > - < 99(0),x("'='^'=)(0) > 

Jo 

T 

< ip{t),Fk{t) > dt 



Taking the limits of both sides, according to the uniform boundedness of the integrals, we 
get, with ^ defined as a distribution on [0, T]: 

< ip{t),i{t) >dt = - [ < ipit),C{t) > dt+ < (^(T), e(T) > - < ip{0),x > 

^° (48) 
= lim / < ip{t),Fk{t) > dt. 

fc— s>oo Jo 

In other words, = limfc_^oo -^fc in the sense of distributions. Moreover, for every T > 0, 
restricting ip to C^([0, T], M") (the set of infinitely differentiable functions from [0,r] to 
R"' with compact support, which is indeed contained in C°°([0, oo), M")) and using the 



density of ([0,T],M") in L'^{[0,T],W) (see e.g. pi]), equation (|48j) also implies that 
the sequence F^ is weakly converg ent in L^{[0,T],R"-). Let us denote by Ft its weak 
limit in L^([0, T], M""). We have therefore constructed a collection {-FT}r>o of weak limits, 
which uniquely defines a function F almost everywhere on the whole interval [0, oo), whose 
restriction to any interval [0, T] coincides a.e. with Ft, i.e. F ^ ^ = Ft a.e.. Indeed, taking 

any pair of intervals [0,Ti] and [0, T2] with Ti < T2, by the uniqueness of the limits Ft^ 
and Ft2, the restriction of Ft2 to the interval [0, Ti] coincides almost everywhere with Ft^ 
and it is readily seen that non uniqueness of F would contradict the uniqueness of every 
Ft. 

By Mazur's Theorem (see e.g. [26^ Chapter V, §1, Theorem 2, p. 120]), for every k, 
there exists a sequence {af , . . . , a^} of non negative real numbers, with J2i=i '^i — 1' such 
that the sequence F^ = J2i=i ca^Fi is strongly convergent to F in every L^([0,r],M") for 
all finite T. Note that this property a fortiori holds true if we replace the sequence Fi by 
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any subsequence Fj constructed by selecting a subsequence of indices ij such that, given 



e > 0, sup(g[o,r] \\f{x^'^*^'^'^\'t): Ui^{t)) — f{(,{t), Ui^{t))\\ < e2 ^ for each j, which is indeed 
possible thanks to the uniform convergence of x^^'"^''^ to ^ and the continuity of /. Note 
also that the limit F remains the same (for convenience of notation, we keep the same 
symbols for the aj's, but we remark that these coefficients have to be adapted relative to 
the new subsequence). 

By Minkowski's inequality, we have: 



< 



||^a^^(/(e(t),n,^.(t))-/(xH'-'.)(t),n,^^(t))) fdt ) (49) 



We now prove that the limits of the two terms on the right-hand side of expression ( 49 ) , 
as k tends to infinity, exist and are equal to 0. The convergence of the second limit to 
is clearly an immediate consequence of the strong convergence of Y^j=i (^j^ij to F. 

For the first term on the right-hand side of ( 49 ) , according to the construction of the 
above subsequence, and using the fact that G [0, 1] for every j, we have 



{f{m,UiM - /(^^"v-^)(t),n,^.(t))) 



<E"'II/(?W'NW)-/(^^"'^''^^-^W'NW)II 

k 

i=i 

k 



(50) 



Thus 



r II E«' {fm.u^^it)) - /(xK'-'.)(t),u.^(t)) 

/° .7 = 1 



(51) 



< [T[e{l-2-^)fY <eVT, 



hence, since e can be chosen arbitrarily small, the left-hand term in (51 ) converges to as 
k tends to infinity. 
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Therefore, the same holds for the left-hand side of (49), which proves that F{t) belongs 
almost everywhere to the closed convex hull of (i))}jeN which is contained in 

f{C{t),Ui) according to (A3). 

Finally, again according to (A3), for every k, there exists, by the measurable selection 
theorem Vk such that 

k 

for almost all t and, since strong convergence implies pointwise convergence almost 
everywhere of a subsequence (see e.g. |15j). we have, from the convergence to of the 



left-hand side of (49), that, taking the limit over such a subsequence. 



k 

lim f{m,Vkit)) = lim ^a,V(e(t),^^i,(0) = a.e. t G [0,oo). 

k^oo fc— >oo — • 

Thus, taking the continuity of / with respect to u into account (see (Al)), we conclude that 
the sequence Vk pointwise converges to some u £U such that F{t) = f {S,{t) , u{t)) for almost 
all t, and therefore that ^ satisfies ^ = /(^u) almost everywhere, with ^(0) = x G Xq. 
By the uniqueness of integral curves of (llj), we conclude that ^(t) = x("'^'^(t) almost 
everywhere and, thus, that ^ £ X, which achieves to prove the lemma. ■ 



B Needle perturbations and the maximum principle (flQl 

[IS [11]) 

B.l Perturbations 

Given u gU and an integral curve x^'^'^\ we consider a non negative real number e G [0, Eq] 
with bounded Eq, an initial state perturbation h £ M" satisfying \\h\\ < H and a variation 
Uk^e of n, parameterized by the vector k = {v, r, /) G C/i x [0, T] x [0, L] with bounded T, L, 
of the form 

A - - { V on [t -1£,t[ , , 

n.,, = nM(._,,)^N.^=| _ ^1^^^^^^^^^ [o^^j (52) 

where v stands for the constant control equal to w G f/i for all t G [r — /e,T[. We also 
consider the corresponding integral curve starting from x + eh and generated 

by liK.e- 

We indeed have = ^{u,x+eh)^^^ ^^^^ t £ [0,r - le[ and, denoting by 

Ze(r — le) = x("'^+^'')(r — /e) and 2;£(r) = a;("'=.='^+^'')(r), we have 



X 
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We also consider the fundamental matrix of the variational equation: 

= <^Hs,s) = In (53) 

where In is the identity matrix of M". 

We have the following approximation result (see e.g. [T9| Chapter II, §13], [161 Chapter 
4, p. 248], PI): 

Lemma B.l The sequence is uniformly convergent to x*^"'^'^ on [0,T] as 

£ tends to 0, uniformly with respect to k and h. 

If, moreover, t is a Lebesgue point of u, we have, for all t € [r, T] ; 

where 

^i;(t,Ac,/i) ^ <l>^(t,0)/i + /cl>^(t,r) (/(x(^'^)(r),t;) - /(x(^'^)(r), 72(t))) . (55) 

The perturbed vector w{t,K,h) at time t associated to k, and h is tangent at x^'^'^\t) to 
the curve of perturbed states e i— )• x^'^'^-'^^'^^^^t) at time t. 

It is immediate to see that multiplying / and h by any real number rj > yields the 
perturbation vector rj ■ w{t, k, h). Therefore, the set of such perturbation vectors forms a 
cone in W^. 



B.2 The perturbation cone 



We now consider an arbitrary sequence of variations of the form (52), with associated 
vector X - {{vi, nJi) : i = 1, . . . , k}, i.e. 



where the perturbation times ri < • • • < are Lebesgue points of u. Lemma B.l still 
applies and, for every t G [rfc,T], the approximation formula (|54[) reads: 



^K.e,x+£fc) _ ^{u,x) ^ ^^(^^ ^) + O(e^) (56) 

where 

w{t, X, h) ^ <I>^(t, 0)/i + X: 4^^(t, n) {f{x^^~^\n),v,) - /(x(^'^) {n)Mn))) ■ (57) 

j=i 

The control variation u^^i, associated to the parameters x £ is called needle perturba- 
tion. The cone generated by convex combinations of such needle perturbations at time t 
is denoted by K-t- Note that we have $"(r2, ri)/CT-i C /Cra for all < n < T2 < T. 
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B.3 The maximum principle 

We recall here the following classical result (see, e.g. Theorem 3, Chapter 4, p. 254]). 

Theorem B.l (Maximum principle) Consider the constrained system Q), 
Let u ^ U be such that x*^"'^'"^(ti) G dXt^{xQ) for some ti > (where dXt^{xQ) denotes 
the boundary of the attainable set defined by Then, there exists a non zero absolutely 

continuous maximal solution rj^ to the adjoint equation 

r{t) = - (^(x(^'-o)(t),n(t)))^r?^(t), (58) 

such that 

max|(??"(t))^/(x("'^«)(t),u)| = (7/"(t))^/(x("'^«)(t),u(t)) = constant (59) 
for almost all t £ [0,ti]. 
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